708878 


Working  Paper  No.  139 


GENERALIZED  BENDERS  DECOMPOSITION 


ARTHUR  M.  GEOFFRION 


April,  1970 


This  <‘.rw  u-r  f--.:  W,  j>,j-  tr,  prcrrcxi 

°T  pu-,;i,:  ,oi-'w  scb; 

: :  *  •* .  •  * :  , 

1  "  •>  -j 


Reproduced  by  the 

CLEARINGHOUSE 

for  federal  Sciontif ic  &  Technical 
Information  Springfield  Va  22151 

WESTERN  MANAGEMENT  SCIENCE  INSTITUTE 


University  of  California,  Los  Angeles  I  "\ 


Il 


GENERALIZED  BENDERS  DECOMPOSITION 
by 

Arthur  M.  Geoffrion 
University  of  California,  Los  Angeles 

April,  1970 

Western  Management  Science  Institute 
Working  Paper  No.  159 


Prepared  for  the  Nonlinear  Programming  Symposium  at  the  University  of 
Wisconsin  sponsored,  by  the  Mathematics  Research  Center,  U.S.  Army, 

May  4-6,  1970. 

This  research  was  supported  by  the  National  Science  Foundation  under 
Grant  OP-8740. 


V 


-i- 


Oeneralized  Benders  Decomposition 
A.  M.  Oeoffrion 

University  of  California,  Los  Angeles 

Abstract 

J.  F.  Benders  devised  a  clever  approach  for  exploiting  the  structure 
of  mathematical  programming  problems  vlth  "complicating  variables".  Such 
problems  have  the  property  that  temporarily  fixing  the  values  of  certain 
variables  renders  the  remaining  optimization  problem  considerably  more 
tractable --perhaps  breaking  apart  into  a  .umber  of  independent  smaller 
problems,  or  being  solvable  by  a  highly  efficient  known  algorithm,  or 
reducing  to  a  convex  program  in  continuous  variables  whereas  the  original 
problem  is  partly  nonconvex  or  discrete.  For  the  class  of  problems  speci¬ 
fically  considered  by  Benders,  fixing  the  values  of  the  complicating 
variables  reduces  the  given  problem  to  an  ordinary  linear  program- 
parameterized,  of  course,  by  the  value  of  the  complicating  variables 
vector.  Hie  algorithm  he  proposed  for  finding  the  optimal  value  of  this 
vector  employs  a  cutting -plane  approach  for  building  up  adequate  repre¬ 
sentations  of  (l)  the  extremal  value  of  the  linear  program  as  a  function 
of  the  parameterizing  vector,  and  (ii)  the  set  of  values  of  the  parameter¬ 
izing  vector  for  vhich  the  linear  program  is  feasible.  Linear  programming 
duality  theory  is  employed  to  derive  the  natural  families  of  "cuts" 
characterizing  these  representations,  and  the  parameterized  linear  pro¬ 
gram  itself  is  used  to  generate  what  are  usually  "deepest"  cuts  for 
building  up  the  representations. 
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In  this  papers  Benders'  approach  la  generalized  to  a  broader  claes 
of  programs  In  vhlch  the  paraaeterlzed  subproblem  need  no  longer  be  a 
linear  program.  Nonlinear  convex  duality  theory  is  employed  to  derive 
the  natural  families  of  "cuts"  corresponding  to  those  In  Benders'  case. 
Die  conditions  under  vhlch  such  a  generalization  is  possible  are  examined 
In  detail  from  both  the  theoretical  and  computational  viewpoints.  An 
illustrative  specialization  is  made  to  the  variable  factor  programing 
problem  Introduced  by  R.  Wilson,  where  it  appears  to  offer  an  especially 
attractive  approach.  A  computational  study  is  currently  under  way  to 
evaluate  the  effectiveness  of  the  generalized  Benders  decomposition 
algorithm  for  several  classes  of  problems. 
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1.  DfTBODUCTION 

nils  paper  is  devoted  to  problems  of  the  form 
(l)  Maximize  f(x*y)  subject  to  G(x*y)  £  0 

x*y  x  €  x 

y  €  Y  * 

where  y  is  a  vector  of  "complicating"  variables  in  the  sense  that  (1) 

Is  a  such  easier  optimization  problem  in  x  when  y  is  temporarily  held 
fixed.  G  is  an  m- vector  of  constraint  functions  defined  on  X  x  Y  c 

nl  °2 

We  have  in  mind  particularly  situations  such  as  the  following: 

(a)  for  fixed  y*  (l)  separates  into  a  number  of  independent 
optimization  problems*  each  involving  a  different  subvector 
of  x 

(b)  for  fixed  y*  (l)  assumes  a  well-known  special  structure 
(such  as  the  classical  transportation  form)  for  which  effi¬ 
cient  solution  procedures  are  available 

(c)  (l)  is  not  a  concave  program  in  x  and  y  Jointly*  but 
fixing  y  renders  it  so  in  x. 

Such  situations  abound  in  practical  applications  of  mathematical  program¬ 
ming  and  in  the  literature  of  large-scale  optimization*  where  tho  central 
objective  is  to  exploit  special  structure  such  as  this  in  order  to  design 
effective  solution  procedures. 

It  is  evident  that  there  are  substantial  opportunities  for  achieving 
computational  economies  by  somehow  looking  at  (l)  in  y-space  rather  than 
in  xy-space.  We  expect  that  in  situation  (a)*  the  computations  can  be 


$ 
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largely  decentralized  and  done  In  parallel  for  each  of  the  smaller 
independent  subproblems;  in  (b),  use  can  he  made  of  available  efficient 
special-purpose  algorithms;  and  in  (c),  the  non-convexities  can  he 
treated  separately  from  the  convex  portion  of  the  problem. 

The  key  idea  that  enables  (l)  to  be  viewed  as  a  problem  in  y-space 
is  the  concept  of  "projection"  [6"|,  sometimes  also  known  as  "partitioning." 
The  projection  of  (l)  onto  y  is: 

(2)  Maximize  v(y)  subject  to  y  f  Y  fl  V  , 

y 

where 

(3)  v(y)  s  Supremum  f(x,y)  subject  to  G(x,y)  £  0 

X  x  f  X 

and 

(4)  V  =  {y  :  G(x,y)  ^  0  for  seme  x  f  X}  . 

Note  that  v(y)  is  the  optimal  value  of  (l)  for  fixed  y  and  that,  by 
our  designation  of  y  as  "coaqplicating"  variables,  evaluating  v(y)  is 
much  easier  than  solving  (l)  itself.  lecause  we  must  refer  to  it  so 
often  in  the  sequel,  we  introduce  the  special  label  (ly)  to  refer  to 
the  optimization  problem  in  (3): 

(ly)  Maximize  f(x,y)  subject  to  G(x,y)  £  o  . 

x  C-  X 

Bie  set  V  consists  of  those  values  of  y  for  which  (ly)  is  feasible; 

Y  n  V  can  be  thought  of  as  the  projection  of  the  feasible  region  of  (l) 
onto  y-space. 
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It  is  Intuitively  clear  that  the  projected  problem  (2)  la  equivalent 
for  our  purposes  to  (l).  This  will  be  made  precise  in  Theorem  1  below. 
For  now,  it  Is  enough  to  keep  in  mind  that  an  optimal  solution  y*  of 
(2)  readily  yields  an  optimal  solution  (x#,y*)  of  (l),  where  x*  Is 
any  optimizing  x  in  (ly*). 

Benders  [l]  was  one  of  the  first  to  appreciate  the  importance  of 
(2)  as  a  route  to  solving  (l).  The  difficulty  with  (2),  however,  is  that 
the  function  v  and  the  set  V  are  only  known  implicitly  via  their 
definitions  (3)  and  (4).  Benders  coped  with  this  difficulty  by  devising 
a  cutting -plane  method  that  builds  up  a  tangential  approximation  to  v 
and  V.  His  development  treats  the  case  in  which  (l)  is  a  "semi linear" 


program: 

(5a) 

X  a  {x  :  x  £  0}  , 

(5b) 

f(x,y)  e  cSc  +  0(y) 

(5c) 

G(x,y)  e  Ax  +  g(y) 

where  0  is  a  scalar-valued  and  g  a  vector -valued  function.  Under  these 
assumptions,  both  v  and  V  turn  out  to  have  exact  representations  usin£ 
only  a  finite  number  of  tangential  approxlmants .  Linear  piogramming 
duality  theory  yields  a  constructive  proof  of  this  result  based  on  the 
fact  that  (l)  is  a  linear  program  in  x  for  each  fixed  y.  The  rationale 
of  the  computational  procedure  is  then  evident.  See  the  original  paper 
or  [6,  Sec.  4.1]  for  details. 
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The  main  result  of  this  paper  is  an  extension  of  Benders'  approach 
to  a  sore  general  class  of  problems.  We  are  able  to  considerably  weaken 
his  semilinearity  assumption  (5)  so  as  to  encompass  additional  problems 
of  practical  Interest.  This  extension  is  made  possible  by  recent  develop¬ 
ments  in  nonlinear  duality  theory  [7]  •  These  results  are  presented  in 
the  following  section.  In  Sec.  3*  specialization  is  made  to  the  Variable 
Factor  Programming  Problem  previously  treated  by  Wilson  [12*1  by  another 
approach.  In  addition  to  its  intrinsic  interest,  this  application  illus¬ 
trates  an  instance  in  which  (l)  is  not  a  concave  program  and  yet  can  be 
solved  by  concave  programming  techniques  within  the  framework  of  the 
present  approach.  Section  4  discusses  the  assumptions  of  Sec.  2  and  some 
alternative  ways  of  handling  the  set  V  so  as  to  achieve  an  algorithm 
that  is  more  primal  in  nature  than  might  otherwise  be  the  case. 

For  the  reader's  convenience,  pertinent  but  not  yet  widely  known 
results  from  nonlinear  duality  theory  are  summarized  in  an  Appendix. 

The  notation  employed  is  standard.  All  vectors  are  columnar  unless 


transposed 
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2.  OEKERALIZED  BENDERS  DECOMPOSITION 

The  generalization  of  Benders'  approach  to  a  class  of  problems  of 
the  form  (l)  Is  divided/  for  pedagogical  reasons/  into  three  subsections. 
The  first  establishes  the  "master"  problem  and  Its  equivalence  to  (l). 

The  central  idea  here  Is  to  invoice  the  natural  outer  tangential  represen¬ 
tations  of  v  and  V  after  passing  to  (2).  The  second  subsection 
demonstrates  how  the  master  problem  can  be  solved  via  a  series  of  "sub¬ 
problems"  which  generate  the  tangential  approxlmairts  of  v  and  V  as 
needed.  Roughly  speaking,  this  is  accomplished  by  obtaining  the  optimal 
multiplier  vectors  for  (ly)  corresponding  to  various  trial  values  of 
y.  The  Generalized  Benders  Decomposition  Procedure  for  (l)  is  then 
stated  and  discussed. 

2.1  Derivation  jf  the  Master  Problem 

The  desired  master  problem  is  obtained  from  (l)  by  a  sequence  of 
three  manipulations: 

(l)  project  (l)  onto  yf  resulting  in  (2), 

(ii)  invoke  the  natural  dual  representation  of  V  in  terms  of 

the  Intersection  of  a  collection  of  regions  that  contain  it/ 

(ill)  invoke  the  natural  dual  representation  of  v  in  terms  of  the 

polntvise  infimum  of  a  collection  of  functions  that  dominate  it. 

Manipulation  (i)  was  already  discussed  in  Sec.  1.  The  following  easy 

theorem  (cf.  [6/  Sec.  2.1])  shows  that  (l)  and  (2)  are  equivalent  for 

our  purposes  (note  that  no  assumptions  on  (l)  whatever  ere  needed) . 
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Theorem  1  (Projection).  Problem  (l)  1b  Infeasible  or  has  unbounded 
optimal  value  if  and  only  if  the  same  is  true  of  (2),  If  (x*>y*) 
is  optimal  in  (l),  then  y*  must  be  optimal  in  (2).  If  y*  is 
optimal  in  (2)  and  x#  achieves  the  sup  remum  in  (3)  vith  y  =  y*, 
then  (x#,y*)  is  optimal  in  (l).  If  y  is  c^-optimal  in  (2)  and 
x  is  c2-optimal  in  (ly),  t1  on  (x>y)  is  (c^+  *2) -optimal  in  (l). 
Manipulation  (ii)  is  based  on 

Theorem  2  (V  Representation) .  Assume  that  X  is  a  nonempty  convex 
set  and  that  G  is  concave  on  X  for  each  1  laoed  y  £  Y.  Assume 
further  that  the  set  Z  =  (z  {  if  :  G(x,y)  =  z  for  some  x  €  X} 

y 

is  closed  for  each  fixed  y  €  Y#  Then  a  point  y  €  Y  is  also  in 
the  set  V  if  and  only  if  y  satisfies  the  (infinite)  system 

(6)  ["supremum  X^foy)]  ^0,  all  X  €  A  * 

x  €  X  J 

m 

where  a  s  (H  if  :  X  «  0  and  E  X,  «  1}. 

i=l  1 

Proof.  Let  y  be  an  arbitrary  point  in  X.  It  is  trivial  to  verify 
directly  that  y  satisfies  (6)  if  it  is  in  V.  The  converse  can  be 
demonstrated  with  the  help  of  nonlinear  duality  theory  as  follows. 

Suppose  that  y  satisfies  (6).  Then 


Infimum  [" Supremum  X^G(x*y)l 
X  €  A  L  x  €  X  J 


It  follows  that 
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since  tne  scaling  of  X  doesn't  influence  the  sign  of  the  bracketed 
expression  and  \  -  0  is  allowed  in  (7).  Nov  (7)  simply  asserts  that 
the  dual  with  respect  to  the  G  constraints  of  the  concave  program 


(8)  Maximize  Ox  subject  to  G(x,y)  >  0 

x  €  X 

has  optimal  value  0.  Recalling  that  Z -  is  closed,  we  therefore  have 

<y 

from  Theorem  A.l  of  the  Appendix  that  (8)  must  be  feasible  and  hence  that 


y  6  V. 


Q.E.D. 


The  assvuqption  that  Z  Is  closed  for  each  y  f.  Y  is  not  a  stringent 

y 

one.  Mild  sufficient  conditions  under  which  it  must  hold  are  given  in 
Sec.  4.1. 

Manipulation  (ill)  is  based  on 

Theorem  3  (v  Representation).  Assume  that  X  is  a  nonempty  convex 

set  and  that  f  and  G  are  concave  on  X  for  each  fixed 

at  least 

y  €  Y.  Assume  further  that,  for  each  fixed  y  €  Y  D  V,  /  one  of  the 
following  three  conditions  holds: 

(a)  v(y)  is  finite  and  (ly)  possesses  an  optimal  multiplier 
vector;-^ 

(b)  v(y)  is  finite,  G(x,y)  and  f(x,y)  are  continuous  on 
X,  X  is  closed,  and  the  e -optimal  solution  set  of  (ly) 
is  nonempty  and  bounded  for  seme  5  *  0; 

(c)  v(y)  =  +®  . 


y See  the  Appendix  for  the  definition  of  an  optimal  multiplier  vector. 
Actually,  it  is  enough  to  consider  generalized  optimal  multiplier  vectors 
(also  defined  in  the  Appendix)  in  order  to  avoid  the  implicit  assumption 
that  (ly)  must  have  an  optimal  solution. 
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Then  the  optimal  value  of  (ly)  equals  that  of  its  dual  on  Y  n  V# 
that  is, 

(9)  v(y)  =  Infimum  I" Supremum  f(x,y)  +  u^Gfoy)  i  ,  all  y  €  Y  fl  V  . 

u>0  LxfX  J 


The  proof  is  a  direct  application  of  Theorems  A. 2,  A. 3,  and  weak 
duality.  Alternative  assumption  (a)  is  one  that  will  very  often  hold. 

Many  different  sufficient  conditions,  usually  called  "constraint  qualifi¬ 
cations,"  are  known  which  imply  it- -most  of  them  fairly  weak  assumptions 
to  preclude  pathological  cases.  Nor  is  alternative  (b)  particularly  strin¬ 
gent.  Of  course,  boundedness  of  X  or  of  the  feasible  region  is  enough 
to  guarantee  that  the  e  -optimal  solution  set  is  bounded  for  any  c  >  0, 
and  the  existence  of  a  unique  optimal  solution  does  it  for  c  =  0. 

Under  the  assumptions  of  Theorems  2  and  3,  then,  manipulations  (i)-(iil) 
applied  to  (l)  yield  the  equivalent  master  problem 


Maximize  ^Infimum  Tsupremum  f(x,y)  + 
y  €  Y  u  >  0  x  €  X 


subject  to  (6) 


or,  using  the  definition  of  infimum  as  the  greatest  lower  bound. 


(10)  Maximize  yn 
y  €  Y  u 

y0 

subject  to 

(10a)  y  £  Supremum  [f(x,y)  +  utG(x,y)l,  all  u  =  0 

x  f.  X 


Supremum  {xt0(x,y)l  *  0,  all  \  €  A  . 
x  €  X 


(10b) 
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2.2  Solving  the  Master  Problem 

The  most  natural  strategy  for  solving  the  master  problem  (10),  since 
it  has  a  very  large  number  of  constraints,  is  relaxation  [6]:  begin  by- 
solving  a  relaxed  version  of  (10)  that  Ignores  all  but  a  fev  of  the  con¬ 
straints  (10a)  and  (10b);  if  the  resulting  solution  does  not  satisfy  all 
of  the  ignored  constraints,  then  generate  and  add  to  the  relaxed  problem 
one  or  more  violated  constraints  and  solve  it  again;  continue  in  this 
fashion  until  a  relaxed  problem  solution  satisfies  all  of  the  Ignored 
constraints  (at  vhich  point  an  optimal  solution  of  (10)  has  been  found), 
or  until  a  termination  criterion  signals  that  a  solution  of  acceptable 
accuracy  has  been  obtained.  Details  regarding  the  termination  criterion 
will  be  supplied  later.  Die  deeper  concern  is  the  crucial  issue  of  hov 
a  solutirr  to  a  relaxed  version  of  (10)  can  be  tested  for  feasibility 
vith  respect  to  the  ignored  constraints  and,  in  case  of  infeasibility, 
hov  a  violated  constraint  can  he  generated. 

A  A 

Suppose  that  (y,yQ)  is  optimal  in  a  relaxed  version  of  (10).  Hov 
can  this  point  be  tested  for  feasibility  in  (10a)  and  (10b)?  From 

A 

Theorem  2  and  the  definition  of  V,  ve  see  that  y  satisfies  (10b)  if 
and  only  if  (ly)  has  a  feasible  solution.  And  if  (ly)  turns  out  to 
be  feasible,  theorem  3  implies  that  (y,yQ)  satisfies  (10a)  if  and  only 
if  yQ  s  v(y).  Thus  (ly)  is  the  natural  subproblem  for  testing  (y,yQ) 
for  feasibility  in  the  master  problem.  This  is  in  perfect  accord  vith 
our  interest  in  applications  vhere  y  is  "complicating"  in  the  sense 
that  (ly)  is  much  easier  than  (l)  itself. 


10 


Not  only  is  (ly)  the  appropriate  subproblem  for  testing  the 
feasibility  of  (y,yQ)  in  (10)j  but  almost  any  reasonable  algorithm 
for  (ly)  will  yield  an  index  of  a  violated  constraint  in  the  event 
that  (y,yQ)  Is  infeasible.  By  an  "index"  of  a  violated  constraint 
we  mean  a  vector  u  £  0  such  that 

A  A  A 

(11a)  y_  >  Supremum  (f(x,y)  ♦  u  O(x,y)l 

xCX 

if  (10a)  is  violated,  or  a  vector  X  €  A  such  that 

(lib)  Supremum  (XtG(x,y)}  <  0 

x  €  X 

if  (10b)  is  violated.  If  (ly)  is  infeasible,  it  can  be  shown  that 

A 

most  dual-type  algorithms  addressed  to  it  yield  such  a  X,  as  do  most 

primal  algorithms  fitted  with  a  "Phase  One"  procedure  for  finding  an 

initial  feasible  solution  if  one  exists  [x  can  be  viewed  as  a  convex 

combination  of  constraints  that  has  no  solution  in  X],  If  (ly)  Is 

feasible  and  has  a  finite  optimal  value,  it  follows  from  Theorem  A.J* 

and  yQ  >  v(y). 

that  an  optimal  multiplier  vector  satisfies  (lla)  if  one  exists  /  Vir- 

A 

tually  all  modern  algorithms  applicable  to  (ly)  produce  an  optimal 
multiplier  vector  as  a  by-product  if  one  exists,  as  1b  usually  the  case. 
Nonexistence  must  be  associated  either  with  an  unbounded  optimal  value, 
in  which  case  one  may  terminate  since  the  same  must  then  be  true  of  (l), 
or  with  a  finite  optimal  value  but  a  pathological  condition  in  which, 

A 

by  (9)>  u  nevertheless  satisfies  (lla)  if  it  comes  close  enough  to 


H 

being  optima],  in  the  dual  of  (ly).  Such  a  u  will  be  referred  to 
subsequently  as  a  "near  optimal"  multiplier  vector. 

In  light  of  this  discussion,  it  is  reasonable  to  presume  henceforth 
that  (ly)  will  be  addressed  with  an  algorithm  that  is  dual-adequate 

A  A 

in  the  sense  that  it  yields:  a  vector  X  €  A  satisfying  (lib)  if  (ly) 

A 

is  infeasible;  an  optimal  multiplier  vector  u  if  one  exists;  or  a  near 

A 

optimal  multiplier  vector  u  satisfying  (lie)  if  no  optimal  multiplier 

A  ^ 

vector  exists  but  yQ  exceeds  the  optimal  value  of  (ly). 

Thus  far  we  have  shown  how  (ly)  can  be  vised  to  test  any  point 

A  A 

(y,yQ)  for  feasibility  in  the  master  problem  (10),  and  to  generate  an 

A  A 

index  (X  or  u)  of  a  violated  constraint  in  the  event  of  infeasibility. 
There  is  still  a  potential  difficulty,  however,  in  carrying  out  the 
relaxation  strategy  as  a  means  of  solving  (10).  Namely,  the  generated 
constraints  may  be  difficult  to  obtain  in  a  form  suitable  for  computa¬ 
tional  purposes  even  though  their  indices  are  known,  since  they  each 
involve  the  optimal  value  of  an  optimization  problem  parameterized  by 
y.  To  overcame  this  difficulty,  it  appears  necessary  to  assume  that 
(l)  has  the  following  property  which,  strong  as  it  appears  to  be,  is  still 
much  weaker  than  Benders'  semilinearity  assumption. 

Property  P.  For  every  u  £  0,  the  supremum  of  f(x,y)  ♦  utG(x,y) 
over  X  can  be  taken  essentially  independently  of  y,  so  that 
there  exists  a  function  L*(  -;u)  on  Y  satisfying 

I»*(yju)  =  Supremum{f(x,y)  +  utG(x,y)l,  y  €  Y 
x  e  x 


(12a) 
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which  can  be  obtained  explicitly  with  little  or  no  more  effort 
than  Is  required  to  evaluate  It  for  a  single  value  of  y. 

Similarly#  for  every  X  €  A#  the  supremum  of  X^x^y)  over 
X  can  be  taken  essentially  Independently  of  y#  so  that  there 
exists  a  function  L#(*;X)  on  Y  satisfying 

(12b)  Xi -.( yj X )  =  SupremmtX ^G( x#  y ) }  >  y  €  Y 

x  €  X 

which  can  be  obtained  explicitly  with  little  or  no  more  effort  than 
is  required  to  evaluate  it  for  a  single  value  of  y. 

One  important  case  in  which  Property  P  holds  is  when  f  and  G  are 
linearly  separable  in  x  and  y: 

f(x,y)  e  f  (x)  +  f„(y)  , 

(13)  1  2 

G(x,y)  =  G^x)  +  Gg(y)  . 

Hien  for  any  u  £  0  and  X  €  A# 

(14a)  L*(y;u)  =  Supremumff., (xJ+uV, (x)l  +  fP(y)*utGp(y),  y  €  Y 

x  €  X  A  x 

(l4b)  L*(y;\)=  SupremunfX^G.  (x) }  +  X^G«(y)#  y  €  Y  • 

x  €  X  A 

The  Variable  Factor  Programming  Problem  discussed  in  Sec.  3  showB  that 
Property  P  can  hold  even  though  F  and  G  are  not  linearly  separable 
in  x  and  y.  See  Sec.  4.1  for  further  discussion. 

A  A 

A  dual -adequate  algorithm  for  (ly)  that  produces  L*(*;u)  or 

A  A  A 

L#(*;X)  essentially  as  a  by-product  as  well  as  u  or  X  will  be 
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* 
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called  L/ dual-ade quate .  In  the  presence  of  Property  P,  L/ dual-adequacy 
usually  requires  very  little  If  anything  more  of  an  algorithm  for  (ly) 
than  dual-adequacy  does.  When  (13)  holds,  for  example,  by  (9)  and  (l*fa) 
we  have 

L*(y;u)  =  v(y)-f2(y)  -  u^y)  +  f2(y)+utGg(y),  y  <=  Y 

*  r 

-when  u  Is  an  optimal  multiplier  vector  [for  then  u  achieves  the 
infimum  in  (9)]*  Oils  equation  is  only  approximate  when  an  optimal 
multiplier  vector  does  not  exist,  but  the  approximation  can  be  made 
arbitrarily  accurate  by  making  u  nearly  enough  optimal  in  the  dual  of 

A  ^ 

(ly)  [cf.  Th.  3]«  Thus  L*(yju)  is  obtained  without  even  having  to 
explicitly  perform  the  supremum  in  (l4a).  A  similarly  satisfactory 
situation  usually  exists  regarding  L#(y;X)>  although  a  detailed  justi¬ 
fication  of  this  statement  would  depend  upon  whether  the  algorithm 

a  A 

produces  \  by  a  Phase  One  or  dual -type  approach  when  (ly)  is  in¬ 
feasible.  In  the  context  of  Variable  Factor  Programming,  it  will  be 
seen  in  Sec.  3  that  any  algorithm  for  (ly)  is  L/ dual-adequate  so  long 

A  ^ 

as  it  produces  an  optimal  solution  x  and  multiplier  vector  u  (both 
of  which  necessarily  exist  for  any  y). 

we  can  now  formally  state  the  relaxation  procedure  for  solving  the 
master  problem  (10).  It  is  assumed  henceforth  that  the  hypotheses  of 
Theorems  2  and  3  hold,  that  Property  F  holds,  and  that  an  L/ dual-adequate 
algorithm  for  (ly)  is  available. 


lb 


Generalized  Benders  Decomposition  Procedure 

Step  1.  Let  a  point  y  in  Y  H  V  be  known.  Solve  the  subproblea 
(ly)  by  any  L/dual-ade quate  algorithm  (terminate  if  the  optimal  value 
Is  +«)  to  obtain  an  optimal  (or  near  optimal)  multiplier  vector  u 
and  the  function  L*(y;u) .  Put  p  -  1,  q  =  0,  *  u,  LED  =  v(y). 

Select  the  convergence  tolerance  parameter  c  >  0. 

Step  2.  Solve  the  current  relaxed  master  problem# 

(15)  Maximize  y.  subject  to  yns  L*(yju^),  j  *  1,  ...,p 

y  €  Y  u  u 

y0  L*(y,‘XJ)  *  0,  J  =  l,...,q  , 

by  any  applicable  algorithm.  Let  (y,yQ)  be  an  optimal  solution.  If 

A 

LED  s  yQ-  s,  terminate. 

Step  3.  Restart  the  L/dual -adequate  algorithm  for  the  revised  subproblem 
(ly).  One  of  the  following  three  cases  must  occur. 

A 

A.  v(zL  =  -Ha  .  Terminate . 

A  A  A 

B.  v(y)  is  finite.  If  v(y)  *  yQ-c,  terminate.  Otherwise, 

A 

determine  an  optimal  multiplier  vector  u  (if  none  exists, 
a  near  optimal  multiplier  vector  satisfying  (lla)  will  do) 
and  the  function  L*(yju).  Increase  p  by  1  and  put 

«  *  A  ^  A  A 

tr=  u.  If  v(y)  >  LED,  put  LED  =  v(y).  Erase  y  and  yQ 
amd  return  to  Step  2. 

a  A 

C.  (ly)  is  infeasible.  Determine  X  in  A  satisfying  (lib), 

and  the  function  L#(yj\)»  Increase  q  by  1  and  put  \. 

A  A 

Erase  y  and  yQ  and  return  to  Step  2. 
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A  few  remarks  are  in  order. 

Remarkl.  It  is  implicitly  assumed  in  the  statement  of  Step  2  that  an 
optimal  solution  of  (15)  exists.  Hiis  problem  is  necessarily  feasible 
in  view  of  the  assumed  existence  of  the  point  y  in  Step  1,  but  to 
preclude  an  unbounded  optimum  during  the  early  executions  of  Step  2  it 
may  be  necessary  to  temporarily  bound  y  or  y^  artificially. 

Remark  2.  Step  1  is  set  up  to  begin  at  a  known  point  y  in  Y  0  V 
because  this  is  believed  to  be  the  situation  most  likely  to  be  encountered 
in  applications.  There  i s  an  advantage  to  utilizing  experience  and  mathe¬ 
matical  or  physical  insight  in  order  to  provide  a  good  initial  feasible 
solution.  If  such  a  point  is  unkjiown,  however,  Step  1  could  be  altered 
in  the  obvious  way  to  accomodate  an  infeasible  initial  point  (cf.  Step  3C). 
An  alternative  would  be  to  make  the  procedure  aB  it  stands  function  in  a 
"Phase  One"  mode  in  order  to  find  a  point  y  in  Y  D  V. 

Remark  3°  The  termination  conditions  can  be  understood  as  follows,  in 
view  of  Theorem  1.  Termination  at  Step  1  means  that  there  is  a  sequence 

<  xu>  feasible  in  (ly)  such  that  <  f(xu,y)  >-.+».  Thus  the  sequence 

<  (xu,y)  >  is  feasible  in  (l)  and  drives  its  value  to  +<*>  .  TeraLnation 
at  Step  3A  is  of  the  same  type.  Termination  at  Step  3B  implies  that  y 

A 

is  « -optimal  in  (2),  for  yQ  is  obviously  an  upper  bound  on  the  optimal 
value  of  (2)  [it  is  the  optimal  value  of  the  equivalent  master  problem 
(10)  with  sene  of  its  constraints  deleted].  Ihus  any  optimal  solution 

A  A  A  a 

x  of  (ly)  yields  an  c -optimal  solution  (x,y)  of  (l)  [if  only  a 
6 -optimal  solution  of  (ly)  can  be  computed,  the  result  is  ( c +6 ) -optimal 
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in  (1)3  •  Termination  at  Step  2  is  similar,  except  that  LED  plays  the 

A 

role  of  v(y);  LED  is  always  set  at  the  greatest  optimal  value  of 
(ly)  found  at  any  previous  execution  of  Step  1  or  Step  3E,  and  so  is 
the  best  known  lower  bound_  on  the  optimal  value  of  (2).  When  LED  yQ-  c, 
the  subproblem  corresponding  to  LED  yields  an  e -optimal  solution  of  (l). 

A 

Note  that  while  the  sequence  of  values  for  yQ  found  at  successive 
executions  of  Step  2  must  be  monotone  nonincreasing,  the  sequence  of 
values  for  v(y)  found  at  Step  3B  need  not  be  monotone  nondecreasing. 

This  is  the  reason  for  introducing  LED. 

Remark  4.  In  practice,  a  reasonably  efficient  algorithm  must  be  available 

for  solving  the  relaxed  master  problem  (15)  at  Step  2.  Except  for  certain 

very  special  cases,  this  normally  requires  that  the  set  Y  be  convex  and 

(sufficient  conditions  are  given  in  Sec.  4.1). 
that  the  L*  and  L^  functions  be  concave  on  Y  /  Then  (15) 

becomes  an  ordinary  concave  programming  problem  to  which  a  number  of 

available  algorithms  apply.  It  should  be  mentioned,  however,  that  the 

case  where  Y  is  a  discrete  set  is  also  of  some  significance.  One  of 

the  main  points  of  Benders'  original  paper  is  that  his  approach  permits 

solution  of  mixed  integer  programs  when  y  is  identified  with  the  integer 

variables  and  x  is  identified  with  the  continuous  variables --provided, 

of  course,  that  a  (nearly)  pure  integer  programming  procedure  is  available 

to  solve  (15). 

Remark  5«  For  computational  efficiency  it  is  important  that  the  algo¬ 
rithms  used  by  the  procedure  to  solve  (ly)  and  (15 )  be  able  to  restart 
each  time  taking  advantage  of  the  most  recent  previous  solution. 


17 


Remark  6.  In  practice,  the  number  of  constraints  in  (15 )  probably 
needn't  be  allowed  to  build  up  indefinitely.  The  important  recent  work 
of  Eaves  and  Zangwill  [3]  and  Topkis  [10,  11]  suggests  that  amply  satis¬ 
fied  constraints  can  usually  be  dropped  after  each  execution  of  Step  £. 

In  the  special  case  where  (10a)  and  (lCb)  are  each  equivalent  to  a  finite 
subset  of  themselves,  as  in  Benders'  case  (5),  a  simple  argument  [6,  Sec.  3*3] 
is  enough  to  justify  dropping  amply  satisfied  constraints  provided  (l)  is 
a  concave  program. 

Remark  7.  It  is  of  same  interest  and  comfort  to  know  that  the  violated  con¬ 
straints  generated  at  Step  3  are  usually  the  most  violated  (or  nearly  so) 
among  all  violated  constraints.  When  u  is  an  optimal  multiplier  vector 
at  Step  3B,  it  follows  from  Theorem  A. 2  that  it  indexes  a  constraint  among 
(10a)  that  is  most  violated  at  (y,yg).  When  no  optimal  multiplier  vector 
for  (ly.  exists,  how  nearly  u  comes  to  indexing  a  most  violated  con- 

A 

straint  depends  solely  on  how  nearly  it  rolves  the  dual  problem  of  (ly). 

A 

Similarly,  how  close  \  comes  to  indexing  a  most  violated  constraint  among 
(10b)  at  Step  3C  depends  solely  on  how  close  it  c cones  to  solving  the  dual 
of  (8)  with  y  equal  to  y  (with  the  dual  vector  normalized  to  a). 

Remark  8.  A  major  potential  difficulty  with  the  procedure  is  that  Step  3C 
may  occur  too  frequently  by  comparison  with  Step  3B,  with  the  result  that 
the  procedure  spends  most  of  its  effort  trying  to  remain  feasible  in  V 
to  the  neglect  of  the  maximization  aspect  of  the  problem.  In  Benders' 
case  (5),  it  turns  out  that  Step  3C  can  never  occur  more  than  a  finite 
number  of  times.  Although  "finite”  can  be  very  large,  at  least  this 
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precludes  Step  3C  from  oc curing  infinitely  often.  In  the  case  studied 
in  Sec.  3,  V  turns  out  to  he  the  vhole  space.  This  means  that  Step  3C 
can  never  occur  at  all.  Thus  the  frequency  with  which  Step  3C  occurs, 
while  It  need  not  he  excessive,  does  have  the  effect  of  limiting  the 
scope  of  problems  for  which  the  procedure  is  effective.  In  Sec.  4.2  an 
attempt  Is  made  to  delimit  this  scope  more  sharply,  and  to  point  out 
modifications  of  the  procedure  that  would  help  to  expand  It. 
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3.  APPLICATION  TO  VARIABLE  FACTOR  PROGRAMMING 

Wilson  [12]  has  defined  the  Variable  Factor  Programming  Problem  as 
follows: 

n2  , 

(l6)  Maximize  Ey.f.(x  ) 

x,y  i=l 

subject  to 

n2  i 

(l6a)  2  x  y.  %  c 

i=l  1 

(l6b)  x1  >  0,  i  =  l,...,ng 

(l6c)  Ay  <  b,  y  £  0  , 


i  1  **2 

where  x  is  an  m- vector  [x  is  the  n^-m- vector  (x  , ...,x  )].  One 

may  interpret  y^  as  the  level  of  activity  i,  and  as  the  profit 

coefficient  or  yield  rate  for  activity  1  when  x*  is  the  allocation 

of  variable  factors  (or  stimulants)  per  unit  of  activity  i.  Constraint 

(l6a)  requires  the  total  allocation  of  variable  factors  to  be  within  an 

availability  vector  c . 

Obviously  (l6)  is  a  proper  generalization  of  the  general  linear 
programming  problem.  As  pointed  out  by  Wilson,  it  arises  in  petroleum 
refining  and  blending  (tetra-ethyl  lead  is  a  stimulant),  certain  pre¬ 
processing  industries,  and  agricultural  planning.  In  the  later  case, 
for  instance,  y^  might  represent  the  number  of  acres  to  be  planted  with 
crop  i,  and  x*  might  lepresent  the  amounts  of  various  kinds  of 
fertilizer,  irrigation  water,  tilling  labor,  etc.,  to  be  used  on  each 
acre  of  crop  i. 


\ 
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It  vlll  be  assumed  that  each  f^  is  a  continuous  concave  function 
on  the  nonnegative  orthant ,  i.e.,  that  the  marginal  productivity  of  the 
variable  factors  Is  nonincreasing.  It  Is  also  assumed  vlthout  loss  of 
generality  that  each  component  of  c  is  positive  (if  some  component  of 
c  were  0,  the  corresponding  variable  factor  could  be  dropped  from  the 
problem).  It  then  follovs  that  the  assumptions  of  Sec.  2  hold.  In  fact, 
a  major  simplification  even  arises  because  the  set  V  turns  out  to  be  all 

no  1 

of  R  .  3his  is  so  because  c  £  0  and  x  =  ...  =  x  =  0  satisfies  (l6a) 
for  any  choice  of  y.  The  assumptions  underlying  the  Generalized  Benders 
Decomposition  Procedure  are  easily  verified  as  follows.  Theorem  2,  of 
course,  is  unnecessary.  In  Theorem  3>  the  initial  assumptions  are  imme¬ 
diate.  Alternative  assumption  (a)  holds  for  all  y  >  0  because  the 
subproblem  (ly),  which  specializes  to 


(I7y) 


x1  £  0, 


Maximize 

*  • .  <  tX  c  f  0 


i=l  1  1 


subject  to 


n2  ±_ 

Z  x  y  S  c  , 
i=l  1 


is  a  concave  program  (remember  that  y  =  0)  with  a  finite  optimal  solu- 

2] 

tion- which  also  satisfies  Slater's  interiority  constraint  qualification 
1  n2 

(x  =  ...  =  x  =  0  is  interior  since  c  >  0).  The  relevant  part  of 


->  The  objective  function  is  continuous, 
that  yA=  0,  and  x*  is  bounded  by  0 


x*  is  arbitrary  for  i  such 
=  x^  5  c/y^  for  i  such  that 


yi  >  0. 
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Property  P,  namely  the  first  part,  also  holds  since  for  all  y  >  0  we 
have 


(18) 


Supremum  (f(x,y)  ♦  u^G^y)! 
x  €  X 


n„ 


n. 


2  2 
Supremum  {  T.  y.  f4(x  )  +  u  (-£  x  y.+  c)} 
1  >  n  °2  >  i=l  *  i=l  1 

X  j-  Uj  •  •  • }  X  =  u 


t 

u 


C  + 


Supremum  ff  (x*)  -  utxi]  =  L*(y;u) 
v1  >  n 


Note  that  L*(y;u)  is  linear  in  y.  Finally,  we  note  that  any  algorithm 
for  solving  (l7y)  at  Step  1  or  3  i6  dual -adequate  so  long  as  it  produces 
an  optimal  solution  x  and  an  optimal  multiplier  vector  u  (the  exis¬ 
tence  of  both  for  any  y  has  been  demonstrated  above).  Such  an  algorithm 
is  also  L/dual -adequate,  because 

(19)  Supremum  {f  (x1)  -  uVl  =  f. (x1)  -  u^x1 

x1  >  0 

holds  for  all  i  such  that  yA  >  0;  for  i  such  that  yi  =  0,  the 
supremum  will  have  to  be  computed.  Thus  L*(y;u)  is  at  hand  directly 
from  (18)  and  (19)  with  little  if  any  extra  work  once  the  subproblem 
(l7y)  is  solved. 

It  would  seem  that  Generalized  Benders  Decomposition  is  particularly 
well  suited  to  Variable  Factor  Programs.  Steps  3^  and  3C  cannot  occur; 
the  relaxed  master  program  to  be  solved  at  Step  2, 
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(20)  Maximum  yn  subject  to  Ay  £  b 

y>0  0 

y0  yQ  s  I/*(ysuJ),  J  -  » 

is  an  ordinary  linear  program  (L*  is  linear  in  y  for  fixed  u);  and 
the  subproblem  (l7y)  to  be  solved  at  Steps  1  and  3  is  a  concave  program 
vith  linear  constraints  and  an  objective  function  that  is  linearly  separ¬ 
able  in  the  xi| s.  This  subproblem  even  has  a  natural  and  meaningful 
interpretation  in  its  own  right  as  the  problem  of  determining  the  optimal 
use  of  variable  factors  given  a  tentative  choice  of  activity  levels.  The 
violated  constraints  generated  are  always  most  violated  constraints  (see 
Remark  7). 

Only  trivial  modifications  of  the  above  analysis  are  required  to 
accomodate  additional  constraints  of  the  form  y  €  Y  and  x*  €  X1,  so 
long  as  Y  and  each  X1  is  convex  and  X1  contains  the  origin.  Of 
course,  (20)  may  no  longer  be  a  linear  program  because  it  would  then 
include  the  constraint  y  €  Y. 


4.  DISCUSSION 


The  first  subsection  discusses  an  assumption  of  Theorem  2  and  some 
of  the  implications  of  Property  P.  The  second  subsection  elaborates  on 
Remark  8  of  Sec.  2  concerning  the  possibility  that  the  Generalized  Benders 
Decomposition  Procedure  may  spend  too  much  effort  on  maintaining  feasibility 
of  the  subproblem.  The  final  subsection  mentions  a  computational  study 
nov  under  vay. 

4.1  Discussion  of  Assumptions  of  Sec.  2 
Theorem  2  assumes  that  the  set 

Z  =  (z  £  H01:  G(x,y)  ^  z  for  some  x  €  XI 
«/ 

must  be  closed  for  each  y  €  Y.  At  first  glance  it  might  seem  that  this 
would  be  so  if  G  were  continuous  on  X  for  each  fixed  y  in  Y  and  X 
were  closed,  but  the  following  example  shows  that  this  is  not  sufficient: 
let  m  =  1,  X  =  [1,®),  and  G(x,y)  =-l/x;  then  clearly  Z  =  (-»,  o), 
which  is  not  closed.  If  X  is  bounded  as  well  as  closed  and  G  is 
continuous  on  X  for  each  y  f  Y,  however,  it  is  easy  to  verify  that 
Z  must  be  closed  for  all  y  €  Y.  It  can  also  be  shown  (with  the  help 
of  Lemma  6  of  [7])  that  these  conditions  remain  sufficient  if  the  bounded¬ 
ness  of  X  is  replaced  by  the  condition:  X  is  convex  and,  for  each 

concave 

fixed  y  €  Y,  G  is  /  on  X  and  there  exists  a  point  z  such 

y 

that  the  set  {x  €  X  :  G(x,y)  ^  z  ]  is  bounded  and  nonempty.  Since  X 

y  concave 

is  required  to  be  convex  and  G  is  required  to  be  /  on  X  for 
fixed  y  f  Y  by  Theorem  3  anyway,  this  condition  is  a  useful  weakening 
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of  the  requirement  that  X  he  hounded.  For  example,  the  condition  holds 
if,  for  each  fixed  y  in  Y,  at  least  one  of  the  component  functions  of 
G  has  a  hounded  set  of  maximizers  over  X  (in  this  case,  select  the 
corresponding  component  of  z  to  he  the  maximal  value  and  let  the  other 

y 

components  he  arbitrarily  small). 

The  significance  of  the  requirement  of  Property  P  that  the  extrema 
in  (12a)  and  (12b)  "can  he  taken  essentially  independently  of  y"  deserves 
further  elaboration.  We  have  already  seen  two  classes  of  problems  for 
which  this  requirement  holds,  namely  linear  separability  in  x  and  y 
[see  (l4)l  and  Variable  Factor  Programming  [see  (18)1.  A  third  class  of 
problems  for  which  it  holds  are  those  in  which  f(x,y)+utG(x,y)  can  he 
written  as  Q(h(x,u),y,u)  for  any  x  £  X,  y  f  Y,  and  u  £  0,  where 
Q  is  increasing  in  its  first  component  and  h  is  a  scalar  function  of 
x  and  u.  Then  clearly 

(21)  L*(y;u)  =  Q(Supremum{h(x,u)},y,u),  y  f  Y  . 

x  €  X 

A  similar  representation  would  also  have  to  he  available  for  X.^G(x,y) 
if  the  second  part  of  P  is  to  hold. 

When  the  achievement  of  the  extrema  in  (l2a)  and  (12b)  is  not  in 
question,  the  following  version  of  Property  P  is  appropriate:  for  every 
u  ^  0  there  exists  a  point  xu  €  X  such  that 

(22a)  Supremum  ff(x,y)»utG(x,y)l  =  f(x  .yj+u^x  ,y),  y  6  Y  , 

x  6  X 

and  for  every  X  £  a  there  exists  a  point  fX  such  that 
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(22b) 


Supremum  (x^foy)}  ®  X^x  ,y),  y  €  Y  . 
x  €  X  A 


Then  L*(y;u)  is  available  directly  from  x  ,  and  L*(y;X)  directly 
from  .  Let  this  version  be  called  Property  P' .  One  of  the  conse¬ 
quences  of  P'  is  that,  by  veak  duality,  the  subproblem  (ly)  must 
have  a  finite  optimal  value  vhe never  it  is  feasible.  Thus  Step  3A  of 
the  Generalized  Benders  Decomposition  Procedure  cannot  occur,  and  termina¬ 
tion  cannot  occur  at  Step  1.  It  is  helpful  to  keep  in  mind  that  the  usual 

A 

situation,  but  by  no  means  the  only  possible  one,  is  where  u  is  an 

A 

optimal  multiplier  vector  for  (ly)  and  x*  is  an  optimal  solution. 

One  conjecture  concerning  Property  P  that  has  probably  already 
occurred  to  the  reader  is  that  the  first  part  implies  the  second  part. 

This  seems  plausible  since  for  any  X  G  A  in  the  second  part  of  P  one 
may  take  u  =  9\  in  the  first  part  with  9  so  large  that  the  influence 
of  f  becomes  inconsequential  by  comparison  with  that  of  SX^,  with 
the  result  that  L#(y;X )  approximately  equals  ^  L  (y;9X)  for  large 
G.  This  conjecture  can  indeed  be  verified  when  X  is  closed  and  bounded 
and  f  and  G  are  continuous  on  X  for  each  fixed  y  f  Y.  The  proof 
is  most  easily  carried  out  in  terms  of  P' .  The  boundedness  of  X  can 
be  weakened  somewhat  along  the  lines  suggested  at  the  end  of  the  first 
paragraph  of  this  subsection. 


Finally,  it  is  appropriate  to  recite  some  sufficient  conditions  for 
L*(L#)  to  be  concave  as  a  function  of  y  for  fixed  u(x).  As 

noted  in  Remark  4  in  Sec.  2,  this  is  usually  necessary  if  the  relaxed 
master  problem  (15)  is  to  be  solvable  by  an  available  algorithm.  Suppose 
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that  Y  is  a  convex  set.  We  consider  several  possible  cases.  When  f 

and  G  are  linearly  separable  in  x  and  y,  it  is  evident  from  (l4) 

that  the  desired  conclusion  holds  if  fg  and  G£  are  concave 

on  Y.  In  Variable  factor  Pro  graced.  ng>  (18)  shows  that  L*  is  even 

linear  on  Y.  When  the  Q- representation  introduced  above  holds*  L*  is 

concave 

concave  if  Q  is  concave  in  its  y -argument,  and  L#  is  /  if  the 
concave 

analog  of  Q  is  /  in  its  y-argument.  And  if  the  slightly  more 
stringent  Property  P'  holds,  (22)  reveals  that  the  desired  conclusion 
obtains  if  f  and  G  are  concave  on  Y  for  each  fixed  x  €  X. 

With  regard  to  this  last  condition,  it  is  perhaps  unnecessary  to  say  that 
marginal  concavity  of  f  and  G  in  x  for  fixed  y  and  in  y 

for  fixed  x  does  not  imply  joint  concavity  on  X  x  Y, 

although  of  course  the  converse  does  hold.  It  is  useful  to  know,  however, 
that  this  stronger  joint  condition  does  imply  without  any  further  quali¬ 
fication  that  L+  and  L*  are  concave  on  Y.  The  proof  follows  the 
lines  of  Theorem  2  in  [61. 

It  is  interesting  to  note  that  under  any  of  these  sufficient  condi¬ 
tions  for  the  concavity  of  L*  and  L*,  not  only  is  (15)  a  concave 

program,  but  so  is  the  original  projected  problem  (2):  by  Theorem  2, 

V  n  Y  is  the  convex  set  [y  €  Y  :  L#(y;X)  *  0,  all  X  €  Al;  and  by 

Theorem  3>  v(y)  =  Infimum  L*(yju)  is  concave  on  Y  fl  V  since  it  is 

u  »  0 

the  Infimum  of  a  collection  of  concave  functions.  Thus  (2)  can  be  a 
concave  program  even  though  (l)  is  not  a  concave  program.  This  is  cer¬ 
tainly  true  in  Variable  Factor  Programming,  as  ve  have  already  pointed 
out.  It  can  also  be  true,  for  example,  when  the  sufficient  condition 
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associated  with  Property  P'  holds. 

4.2  On  Maintaining  Feasibility  in  the  Subproblam 

As  discussed  in  Renark  8  of  Sec.  2,  the  effectiveness  of  the  Gen¬ 
eralized  Benders  Decomposition  Procedure  is  diminished  if  8tep  3C  arises 
too  often,  i.e.,  if  the  solutions  of  the  relaxed  master  problem  at  Step  2 
too  often  lie  outside  of  V.  It  was  pointed  out  that  this  difficulty 
will  not  materialize  for  applications  such  as  Variable  Factor  Programming 
in  which  Y  c  V.  In  other  applications  it  may  be  possible  to  specify, 
based  on  prior  experience  or  by  physical  or  mathematical  insight,  a 
region  entirely  within  V  known  to  contain  an  optimal  solution  of  (2). 
This  information  would  be  used  to  redefine  Y  so  that  Y  c  V  holds. 

If  this  cannot  be  done,  sometimes  it  is  still  possible  to  obtain  V 
explicitly  before  commencing  the  procedure.  This  possibility  is  mainly 
limited  to  cases  in  which  G  or  y  is  of  very  low  dimensionality.  For 
example,  if  m  =  1  (i.e.,  G  is  a  scalar-valued  function)  then  V  can 
be  represented  by  the  single  constraint  L#(y;l)  *  0.  The  case  =  1 
(i.e.,  y  is  a  scalar)  is  also  manageable  when  (2)  is  a  concave  program— 
see  the  sufficient  conditions  given  in  Sec.  4.1— because  then  V  is  a 
convex  subset  of  the  real  line  and  must  therefore  be  a  simple  interval. 

It  would  not  be  difficult  to  determine  the  endpoints. 

Step  3C  will  not  occur  at  all  in  the  above  cases  because  there  was 
no  need  to  approximate  V.  The  next  best  situation  is  where  Step  3C  is 
limited  to  a  finite  number  of  possible  occurrences  because  the  represen¬ 
tation  of  Theorem  2  is  essentially  a  finite  one.  This  is  so  (and  he 
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recognized  it)  under  Benders'  assumptions  (5a)  and  (5c): 


Supremum  k[Ax  +  g(y)  -  b] 
x  >  0 

=  X  [g(y)  -  b]+  Supremum  (X  A)x 
x  s  0 


=  xt[g(y)  -  b]  +  \ 


♦-  if  X*A  i  0 
0  if  x*A  <  0  . 


It  follovB  that  V  is  represented  by  the  finite  system 

(23)  (xJ)t[g(y)  •  b]  i  0,  ,)  =  l,...,r  , 

f  1  p 

where  [X  ,...,X  }  is  the  set  of  extreme  points  of  the  convex  polyhedron 

+  » 

(24)  {X  £  0  :  XxA  i  0  and  E  X,  =  1) 

i-1  1 

(if  this  polyhedron  is  empty,  then  V  =  R  ).  Provided  the  L/dual-adequate 
algorithm  selected  for  Step  3  is  arranged  so  as  to  generate  only  extreme 
points  of  the  polyhedron  (24),  Step  3C  can  occur  no  more  than  r  times. 
Hopefully  only  a  small  subset  of  all  r  extreme  points  will  ever  have 
to  be  generated.  If  this  is  not  the  case,  however,  then  it  may  be  wise 
to  modify  the  procedure  so  as  to  essentially  absorb  Step  3C  into  Step  2 

a 

so  that  priority  is  given  to  assuring  that  y  6  V  when  the  relaxed  master 
problem  is  solved.  Ibis  can  be  done  if  (i)  a  parametric  programming 
algorithm  for  the  subproblem  (ly)  is  available  that  can  accomodate 
linear  perturbations  of  y  [4,  8,  9],  and  (ii)  a  feasible  directions 
algorithm  is  used  for  the  relaxed  master  problem  (15).  Since  most 


feasible  directions  algorithms  require  knowledge  only  of  the  constraints 
binding  (or  very  nearly  binding)  at  the  current  feasible  solution  and 
alter  this  solution  by  a  sequence  of  linear  moves,  it  is  possible  to 
carry  out  the  feasible  directions  algorithm  for  (15)  with  q  =  r  without 
having  to  generate  all  r  extreme  points  in  advance.  Hie  relevant  extreme 
points  would  be  generated  as  needed  by  the  parametric  algorithm  for  (ly), 
which  would  track  the  optimal  solution  of  the  subproblem  during  each 
linear  move  of  y  and  produce  an  appropriate  new  constraint  [extreme 
point  of  (24)1  whenever  (ly)  is  about  to  go  infeasible  (i.e.,  whenever 
y  is  about  to  leave  V).  Space  limitation,  preclude  giving  further 
details  here. 

There  may  be  other  useful  conditions  under  which  V  has  a  finite 
representation,  but  in  general  (10b)  could  be  an  infinite  system  with 
the  result  that  Step  3C  may  occur  infinitely  many  times  in  a  row.  In 
this  event  one  may  have  to  abandon  the  procedure  entirely  unless  a  modi¬ 
fication  such  as  the  following  is  sufficiently  effective  in  circumventing 
the  difficulty.  Only  a  brief  outline  can  be  given  here  of  the  suggested 
modification,  which  would  be  to  treat  the  set  Y  Pi  V  in  (2)  by  inner 
rather  than  outer  approximation,  assuming  that  it  is  convex.  In  the 
terminology  of  [6],  the  idea  would  be  to  solve  (2)  by  outer  linearization/ 
relaxation  applied  to  the  objective  function  v  only  (instead  of  to  both 
v  and  Y  P  V),  with  the  relaxed  master  problems  solved  by  inner  lineari¬ 
zation/restriction  applied  to  the  set  Y  D  V  only.-^  This  possibility 

Inner  lineariiation/restriction  Jo  a  conceptual  approach  due  to  Dantzig 
and  Wolfe  [2,  Ch.  241.  See  [6,  Sec.  4.3]  for  the  extension  of  this  approach 
needed  here. 
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was  examined  and  seemed  quite  attractive  in  [5,  Sec.  3»11  for  a  structure 
in  vhich  G  was  separable  in  x  and  y  and  linear  in  y.  The  advantage 
is  that  the  new  relaxed  master  problem, 

(25)  Maximize  y 

y  €  Y  n  V 

y0 

subject  to  yQ  s  L*(y;uJ),  J  =  1, ...,p  , 

A  A 

would  be  solved  pri  mai  i  y  for  y;  thus  y  would  necessarily  be  in  V 
end  go  (ly)  could  not  be  infeasible.-^  The  price  paid  for  eliminating 
Step  3C  entirely  in  this  way  is,  of  course,  more  work  at  Step  2. 

4.3  Computational  Experience 

A  computational  study  is  now  under  way  to  compare,  for  several  classes 
of  test  problems,  the  efficiency  of  Generalized  Benders  Decomposition 
against  a  direct  nonlinear  programming  attack.  A  single  sophisticated 
nonlinear  programming  algorithm  (NONLIN,  due  to  Prof.  G.  W.  Graves  at 
UCLA)  is  being  used  both  for  the  direct  attack  and  for  the  relaxed  master 
problems  ar.u  the  subproblems .  The  results  of  this  study  will  be  reported 
at  a  later  date. 


— ^In  practice,  the  inner  linearization/ restriction  algorithm  applied  to 
(25)  would  be  terminated  suboptimally,  especially  during  the  early  execu¬ 
tions  of  Step  2  (p  small).  Since  an  upper  as  veil  as  lower  bound  is 
available  to  use  in  a  termination  criterion  for  (25),  hovever,  it  is  a 
simple  matter  to  increase  the  required  accuracy  at  successive  executions 
so  as  to  maintain  control  over  the  convergence  of  the  entire  procedure. 
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APPENDIX 

Summary  of  Relevant  Results  from  Nonlinear  Duality  Theory 


All  results  -will  be  stated  in  terms  of  the  standard  primal  problem 


(A.l)  Maximize  f(x)  subject  to  g.  (x)  £  0,  i  =  1,  ...,m  > 

x  €  X  1 

vhere  it  is  assumed  that  f  and  each  function  is  concave  on  the  non¬ 
empty  convex  set  X  E  Rn.  See  CTl  for  further  details. 


The  dual  of  (A.l)  with  respect  to  the  g.^  constraints  is 

(A. 2)  Minimize  Tsupremum  f(x)  +  ”  u,g.(x)1  , 

u  =  0  xfX  i=l  1  1  J 

where  u  =  (i^,  ...,um)  is  a  vector  of  dual  variables.  By  the  Weak 
Duality  Theorem  [ibid.,  Th.2],  any  feasible  solution  of  the  primal  must 
have  a  value  no  greater  than  the  value  of  any  feasible  solution  of  the 
dual. 

If  x  is  an  optimal  solution  of  the  primal,  an  optimal  multiplier 

vector  u  is  defined  to  be  any  nonnegative  vector  satisfying: 
m  _  _  _  m  _ 

E  a.g. (x)  =  0  and  x  maximizes  f(x)  ♦  E  u.g.(x)  over  X.  To 
i=l  11  i=l  1  1 

preclude  assuming  that  the  optimal  value  of  the  primal  is  actually 

achieved  by  some  x,  for  many  purposes  it  is  enough  to  work  with  the 

concept  of  a  generalized  optimal  multiplier  vector:  a  nonnegative  vector 

u  such  that,  for  every  scalar  c  >  0,  there  exists  a  point  x  feasible 

€ 

in  the  primal  problem  satisfying  the  two  conditions  (i)  x  is  an 

m  _  * 

s-optimal  maximizer  of  f(x)  +  E  u.g.  (x)  over  X,  and 

i=l  1  1 
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m  _ 

(ii)  Eu.g.(x,)  s  €  .  Every  optimal  multiplier  vector  is  also  a 
i-1  1  1 

generalized  optimal  multiplier  vector. 

Theorem  A.l.  If  {z  6  H01  :  g^x)  as  z^  i  =  1, ...,m,  for  some  x  €  X) 
ie  closed  and  the  optimal  value  of  the  dual  is  finite,  then  the  primal 
problem  must  be  feasible. 

Proof.  This  is  an  immediate  corollary  of  Theorem  5  [ibid.]. 

Theorem  A. 2.  If  u  if  a  ( generalized)  optimal  multiplier  vector  for  the 
primal  problem,  then  u  is  also  an  optimal  solution  of  the  dual  and  the 
optimal  values  of  primal  and  dual  are  equal. 

Proof.  This  follows  immediately  from  Lemmas  3  and  4  [ibid.l,  and  the 
discussion  Just  before  the  latter. 

It  is  also  true  that  if  the  primal  has  no  generalized  optimal  multi  - 
plier  vector  and  yet  the  optimal  values  of  primal  and  dual  are  equal, 
then  the  dual  can  have  no  optimal  solution  (by  inspection  of  Diagram  1 
[Ibid.l). 

Theorem  A. 3.  Assume  X  to  be  closed,  f  and  each  to  be  continuous 

on  X,  the  optimal  value  of  (A.l)  to  be  finite  and  the  set 

(x  (?  X  :  gi(x)  iO,  i  =  1,  ...,m,  and  f(x)  £  a] 

to  be  bounded  and  nonempty  for  some  scalar  a  no  greater  than  the  optimal 
value  of  (A.l).  Then  the  optimal  values  of  the  primal  and  dual  problems 
are  equal. 

Proof.  This  follows  immediately  from  Theorems  7  and  8  [ibid.]. 
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